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SECTION  0.0  - SUMMARY 


In  thin  investigation,  which  wus  limited  to  ground  bused  electronic  systems, 
the  qualitatively  well-known  relationships  between  reliability  expenditures  in  the 
development  phase  and  achievod  reliability  (in  terms  of  me  an -time -be  tween  failures, 
MTBF)  were  quantified. 

Reliability  program  expenditures  were  divided  into  three  phases:  design,  parts 
and  evaluation.  Three  major  areas  of  quantification  were  investigated  and  models 
developed. 

First,  mathematical  models  were  developed  for  predicting  total  reliability 
expenditures  and  individual  phase  reliability  expenditures.  These  models  were 
based  on  readily  available  independent  variables. 

Second,  reliability  gain  was  developed  for  each  reliability  program  phase  and 
for  the  total  reliability  program.  Models  were  then  developed  which  predict  relia- 
bility gam  based  on  reliability  expenditures.  This  was  accomplished  on  a phase  and 
a total  program  basis. 

Finally,  the  optimal  expenditure  of  reliability  effort  was  investigated.  It  was 
found  that,  in  the  present  state  of  the  art,  the  major  reliability  effort  should  be 
expended  in  the  evaluation  (developmental  and  demonstration  testing)  phase.  This 
result  is  something  of  a discouraging  comment  on  the  "way"  reliability  operates  at 
the  present  time:  we  do  not  know  how  to  expend  reliability  effort  efficiently  in  the 
design  phase,  it  is  hoped  that  this  will  change  in  the  future  since  building  reliability 
into  a product  is  cheaper  than  inspecting  it  in. 
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SECTION  1.0  - INTRODUCTION 


1.1  Fuipose  of  the  Study 

It  is  now  well-known  that  system  reliability  materially  affects  system  life  cycle 
costs.  The  typical  Reliability  Program,  with  its  constituent  parts:  the  design 
phase,  the  parts  phase  and  the  evaluation  phase,  is  a powerful,  useful  tool  in 
achieving  reliable  systems.  It  is  also  well-known  that  the  Reliability  Program 
often  involves  considerable  expenditure  in  the  development  (sometimes  called 
the  acquisition)  phase.  Not  a great  deal  has  been  done  toward  quantifying  the 
relationship  between  Reliability  (development)  Program  costs  and  reliability. 
The  first  definitive  work  along  these  lines  is  ref  [1J.  In  ref  [1]  such  relation- 
ships were  studied  and  quantified  for  airborne  systems.  The  general  purpose  oi 
this  present  study  was  to  quantify  the  relationships  between  Reliability  (develop- 
ment) Program  costs  and  reliability  factors  for  ground  equipment. 

Specifically,  there  were  three  objectives: 

i)  Develop  prediction  equations  for  final  (achieved)  MTV.  in  terms  of 
reliability  costs  (total  and  phase)  and  parts  counts,  and  develop  prediction 
equations  for  reliability  costs  (total  and  phase)  in  terms  of  parts  counts  and 
final  MTBF. 

ii)  Determine  the  reliability  gain  for  each  phase  (design,  parts,  and 
evaluation)  and  develop  prediction  equations  for  this  gain  in  terms  of  the 
costs  of  the  phases. 

iii)  Investigate  the  problem  of  allocating  reliability  funds  in  an  optimal  manner 
among  the  various  phases  of  the  Reliability  Program. 

In  the  next  section  we  give  the  approach  used  in  accomplishing  the  afore- 
mentioned objectives. 

1„2  The  Study  Approach 

In  order  to  accomplish  the  objectives  listed  in  subsection  1.1,  the  following 
tasks  were  set  up  and  accomplished.  These  tasks  are  described  briefly  below. 

1)  Build  the  Data  Base 

Data  are  not  easy  to  find  because  reliability  costs,  by  phase,  are  difficult 
to  obtain.  An  intensive  search  led  to  a data  base  of  ten  (10)  ground-based 
systems  of  widely  varying  types  and  of  relatively  recent  vintage. 

2)  Data  Evaluation  and  Normalization 

The  data  went  through  a final  evaluation  to  make  certain  that  all  the 
required  parameters  (e.g. , total  Reliability  Program  costs,  phase  costs, 
predicted  MTBF,  specified  MTBF,  phase  MTBF's,  parts  costs,  etc.)  were 
available.  The  data  was  normalized  to  eliminate  effects  which  would  con- 
fuse the  results.  For  example,  (use)  environmental  effects  were  largely 
eliminated. 
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3)  Select  Models 


A number  of  mathematical  models  were  selected  as  possible  good  models 
for  use  as  prediction  equations.  Denoting  for  the  present,  a particular 
dependent  variable  as  Y and  a particular  independent  (prediction)  variable 
as  Xi  five  models  were  tried  on  every  data  set  (i.e.,  ever,  model  was 
fitted  to  each  case).  The  models  used  were  («  represents  the  random  error 
term): 

Linear 


Y = a_  + u.X  + ...  + a X + * 
Oil  n n 


Ln- Linear 


InY  - ln  aQ  + 


a.  In  X,  + ...  + a lnX  + « 
11  n n 


Exponential 


Y * .<a°  * *1*»  + 


+ a X + «) 
n n 


Second  Degree 

Y = a.  + a X + a,„X?  + ...  + a .X  + a „X2  +« 

0 11  1 12  1 nl  n n2  n 

Second  Degree  With  Cross  Products  (example,  for  n = 2) 

Y = 30  + ailXl  + ai2Xl  * a2!X2  + a22X2  + a(12)XlX2  + 8 


These  models  were  selected  because  they  are  feasible,  simple,  and  easy  to 
fit  (by  least  squares).  The  linear  and  ln-  linear  models  were  used  exten- 
sively in  ref  [1]. 


4)  Goodness  of  Fit  of  Models 

For  applications  of  the  results  of  this  study  only  one  of  the  above  models  is 
needed  for  any  particular  set  of  variables  (Y,  Xj, . . . , Xn).  Also,  it  is  of 
interest  to  see,  over  a variety  of  situations,  whether  one  particular  model 
is  invariably,  or  even  frequently,  the  best  fitting  model. 

Because  of  the  absurdity  of  the  assumption  that  any  particular  data  set  (of 
dependent  and  some  independent  variables)  is  a random  sample  from  a 
multivariate  normal  distribution,  the  usual  measures  of  goodness  of  fit 
(F  test,  t test  and  Correlation)  have  been  abandoned.  The  two  measures  of 
goodness  of  fit  which  we  have  selected  are  R and  R.  E..  The  formal  defini- 
tions of  these  quantities  are  given  in  the  next  section.  In  words,  R 
measures  the  average  (arithmetic  mean)  absolute  value  of  the  relative  (to 
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the  observed  values)  deviation  of  the  observed  and  calculated  (from  the 
model)  values  of  the  dependent  variable.  R.  E.  measures  the  fraction  of 
the  unexplained  variation  to  the  total  variation.  The  smaller  the  values  of 
R and  R.  E.  for  a particular  data  set  the  better  the  lit.  The  ideal,  but 
impossible,  situation  would  be  R =»  0 and  R.  E.  = 0. 

1.3  Definitions 

CD  - Reliability  design  phase  cost  (in  man-days) 

C - Relative  reliability  design  phase  cost, 

- Reliability  evaluation  phase  cost  (in  man-days) 

L 

C'  - Relative  reliability  evaluation  phase  cost,  C1  = C /C_ 

L c.  t.  1 

Cp  - Reliability  parts  phase  cost  (in  man-days) 

Cp  - Relative  reliability  parts  phase  cost,  Op  = Cp/C^ 

CT  - Total  cost  of  reliability  program  (in  man-days) 

Gp-  Reliabil'tv  gain  due  to  reliability  design  effort,  GD  = 

Gp  - Reliability  gain  due  to  reliability  evaluation  effort,  GE  = ©E/0p 
Gp  - Reliability  gain  due  to  reliability  parts  effort,  Gp  * ©p/©D 
G^,  - Reliability  gain  due  to  total  reliability'  program,  G^,  - gdcpge  - Vei 
k-factor  - Adjustment  factor  for  environmental  applications 

N - Total  number  of  digital  and  analog  parts,  N = + NQ 

N.  - Total  number  of  system  analog  parts 

t\ 

Npj  - Total  number  of  system  digital  parts 

N - Number  of  system  parts  normalized  to  analog 
hA 

©P  - Post  design  MTBF  (in  hours) 

© - Post  evaluation  MTBF  (in  hours) 

£ 

©j  - Initial  system  MTBF  without  reliability  enhancement  (in  hours) 

©p  - Post  parts  MTBF  (in  hours) 

a , - Predicted  MTBF  (in  hours) 
upred 

6 - Specified  MTBF  (contractual  (in  hours) 

8 pec 


4 


100 


where 

Y.  = the  ith  observed  value  of  a particular  function. 
/\  th 

Y = the  i calculated  value  of  a particular  function 
n = number  of  observations. 
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(calculated  from  a model). 

*rl 


J 


I 


I 


SECTION  2.0  - DATA  SOURCES,  COLLECTION,  AND  EVALUATION 
2.1  Data  Sources 


2.1.1  Internal 

Due  to  the  proprietary  nature  of  the  data  required  for  the  generation  of  the 
dita  base,  only  sources  internal  to  the  Hughes  Aircraft  Company  could  be 
utilized  to  construct  the  data  base.  Permission  to  access  program  cost  docu- 
mentation and  labor  records  was  provided  by  the  various  program  offices. 

2.1.2  External 

A literature  search  by  the  National  Aeronautics  and  Space  Administration  and 
by  the  Defense  Documentation  Center  yielded  no  reliability  cost  information 
that  could  be  of  use  for  the  data  base  structure.  Generally,  such  data  was  not 
detailed  enough  to  be  of  use. 

One  report,  ref - [ 1J , was  an  earlier  work  that  studied  the  relationships  of  cost 
versus  reliability  for  airborne  systems.  This  report  was  closely  studied  for 
possible  application  of  the  approach  methodology  although  the  actual  data  was 
not  applicable. 

2.2  Data  Collection 

2.2.1  Summary  of  data 

The  data  for  the  systems  investigated  for  the  study  comprise  two  categoriec. 
The  first  category  contains  system  characterization  and  reliability  data.  This 
data  gathered  on  each  system  expresses  the  type  and  function  oi  the  system, 
the  reliability  values  in  terms  of  mean  time  between  failure  (MTBF)  for  the 
contractual  specified  (©0  = 6spec)»  the  design  predicted  (©pred),  and  the 
demonstrated  (©t|emo)  reliability  values,  and  the  system  complexity  as 
defined  by  the  number  of  total  parts  (N).  The  number  of  parts  excludes  hard- 
ware and  equipment  that  had  no  direct  effect  on  the  system  reliability. 

The  number  of  parts  was  also  represented  by  the  total  number  of  system  ana- 
log parts  (Na)  and  the  total  number  of  system  digital  parts  (Np).  This  data  is 
summarized  in  the  System  Characterization  Table,  Table  2.2.1.  The  second 
category  contains  the  reliability  cost  data.  The  reliability  program  was 
defined  by  determining  the  reliability  program  phases  (see  section  2.4.1)  and 
grouping  the  reliability  program  costs  under  the  three  program  phases.  The 
reliability  program  phase  costs  are  the  reliability  design  phase  costs  (Cp), 
the  reliability  parts  phase  costs  (Cp),  and  the  reliability  evaluation  phase 
costs  (Cp).  The  total  reliability  program  costs  (C-p)  are  the  summation  of  the 
three  reliability  program  costs  and  all  cost  values  are  expressed  in  man- 
dayo.  This  data  is  summarized  in  the  Reliability  Program  Phase  Costs 
Table,  Table  2.2,2. 
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TABLE  2.2.2.  Reliability  Program  Phase  Costa 


System  Number 

Design  Phase 
CD 

(man-days) 

Parts  Phase 

Cp  V 

(man-days) 

Evaluation  Phaso 
C» 

(man-days) 

Total 

C>j> 

(man-days) 

1 

214 

4962 

2498 

7674 

2 

244 

5301 

2196 

7741 

3 

202 

4093 

2249 

3544 

4 

207 

4467 

2464 

7138 

5 

237 

3580 

1348 

5165 

6 

204 

7233 

9262 

16699 

7 

170 

1612 

530 

2312 

8 

119 

3396 

1452 

4967 

9 

272 

1252 

928 

2452 

10 

148 

1449 

2110 

3707 

2.2.2  Datacenter 

The  Systems  Effectiveness  Department  of  Hughes  Aircraft  Company's  Ground 
Systems  Group  maintains  a Reliability/ Maintainability  data  center.  The  data 
center  contains  all  reliabi1  ity /maintainability  information  gathered  during 
in-house  development,  environmental,  and  field  use  on  all  Ground  Systems 
Group  programs.  The  data  center  provided  all  of  the  reliability  data  and  sys- 
tem characterization  information  for  most  of  the  systems  selected  for  the  data 
base.  The  data  was  extracted  from  proposals,  contract  CDRLs,  Interdepart- 
mental Correspondences,  and  system  final  reports.  The  sources  are 
ref.  [2],  [31,  [4],  [5],  [6],  [7J,  [8],  [9],  [10],  [11]. 

2.2.3  Intra-company  contacts 

The  acquisition  of  the  cost  data  for  the  reliability  program  activities  proved  to 
be  both  tedious  and  time  consuming.  The  reliability  program  phases  (see 
section  2.4.1)  were  determined  in  order  to  first  define  the  actual  efforts 
involved  In  a reliability  program  and  second  to  categorize  those  efforts  that 
enhance  system  reliability  during  a particular  phase  of  the  reliability  pro- 
gram. Once  the  reliability  program  phases  were  defined  and  the  reliability 
tasks  assigned  to  one  of  the  phases,  the  acquisition  of  the  cost  data  was 
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concentrated  to  particular  divisions  and  groups  that  were  responsible  for  the 
individual  reliability  program  tasks. 

2.3  Data  Organization 

2.3.1  {System  characterization 

Data  were  surveyed  and  gathered  only  on  systems  that  conformed  to  inhabited 
fixed  ground,  mobile  ground,  or  shipboard  system  types.  All  of  the  systems 
inv  ‘stigated  and  those  that  were  used  to  compile  the  data  base  had  reliability 
programs  that  conformed  to  or  were  modifications  of  MIL- STD-785  A, 
"Reliability  Program  for  Systems  and  Equipment  Development  and 
Production".  This  standard  was  a contractual  requirement  to  insure  that  the 
system  reliability  would  achieve  the  overall  program  objectives  and  meet  the 
contractual  reliability  requirements. 

The  ten  systems  that  comprise  the  data  base  provide  a sample  of  vailed  sizes, 
complexities,  and  applications.  The  system  types  range  from  a small  single 
cabinet  data  MODEM  unit  to  a highly  complex  portable  tracking  and  control 
center.  The  environmental  applications  range  from  fixed  ground  shelters  for 
some  of  the  systems  to  inside  a combat  tank  for  another  system.  For  most  of 
the  systeii  s the  environmental  stresses  are  less  severe  than  what  would  be 
experienced  for  airborne  or  space  applications.  The  systems  generally  have 
adequate  ventilation,  the  temperatures  are  not  excessive  because  spaces  are 
inhabited,  and  the  particular  design  volumes  are  not  severely  constrained  so 
design  volume  is  not  a critical  factor. 

2.3.2  Program  time  period 

Only  systems  that  were  developed  during  the  preceding  ten  years  were 
considered  as  candidate  systems  for  the  data  base.  The  development  time 
spans  for  the  systems  are  represented  in  the  System  Development  Time  Span 
table.  Table  2.3.1.  Although  the  data  center  contained  equipment  characteri- 
zation and  reliability  data  on  systems  that  were  developed  as  much  as  twenty 
years  ago,  the  state  of  the  art  technology  has  experienced  such  rapid  growth 
during  that  time  that  the  system  designs  and  individual  component  type  relia- 
bilities are  not  compatible  with  more  current  systems.  Additionally,  reliabil- 
ity cost  data  doe3  not  exist  for  very  early  systems  due  to  systematic  file 
updating  and/or  destruction  of  obsolete  program  files. 

2.4  Data  Base  Categories 

2.4.1  Reliability  program  phases 

Since  the  overall  reliability  program  plans  differed  from  program  to  program 
due  to  system  application  or  contractual  requirements/objeclives,  a standard 
reliability  program  plan  had  to  be  determined  so  that  the  reliability  program 
plan  for  all  the  data  base  systems  would  be  in  common  terms.  To  simplify 
the  standard  reliability  program,  and  yet  match  the  available  data,  three 
categories  of  reliability  effort  were  defined.  Each  category  was  a time-'oased 
phase  of  the  reliability  program  where  specific  reliability  efforts  for  the 
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phase  would  reflect  an  Improvement  of  the  final  (post  development)  system 

reliability.  The  reliability  program  phase  categories  are  defined  as  follows; 

1)  Reliability  Design  Phase  - the  reliability  tasks  that  comprise  this  phase  are 
design  review  support,  developing  system  reliability  models  and  calcula- 
ting the  predicted  system  reliability  during  various  stages  of  the  program 
design  phase,  determining  reliability  allocations,  and  performing  critical  - 
'ty  studies  and  failure  mode  and  effects  analyses  (FMEA).  A system  reli- 
ability model  is  used  to  ^flne  the  mission  use  of  the  system,  identify  the 
operational  modes  of  the  subsystems,  and  model  system  life.  The  criti- 
cality study  in  conjunction  with  the  FMEA  identifies  potential  system  weak- 
nesses and  critical  items  that  significantly  affect  the  ability  of  the  system 
to  successfully  perform  if  failure  occurs.  The  reliability  prediction  that 
is  performed  at  various  system  design  development  phases  is  an  indica- 
tion of  the  achievable  MTBF  of  the  system.  The  prediction  methodology 
includes  the  stress  factors  determined  by  the  operational  and  environmen- 
tal conditioas  that  a flee  t the  failure  rates  of  the  system  comroncnts  and 
compt  <es  the  individual  failure  rates  up  to  the  circuit  module  level.  All  of 
the  module  failure  rates  are  then  used  in  the  system  reliability  model  to 
determine  the  compatibility  of  the  system  design  MTBF  with  the  contrac- 
tual MTBF.  The  design  reviews  are  conducted  periodically  to  present  the 
results  of  the  reliability  design  assessment  with  Engineering  and  other 
participating  organizations  to  determine  if  the  contractual  objectives  of  the 
program  are  being  achieved.  Alternative  design  approaches  are  also 
assessed  and  tradeoffs  in  design  are  studied. 

2)  K«  liability  Parts  Program  Phase  - The  reliability  tasks  that  make  up  the 
parts  prog  phase  are  parts  stuntiurduatiou  and  selection  according  to 
preierred  parts  lists,  parts  screening  am.  vendor  controls  for  purchased 
parts,  component  and  circuit  stress  rests  and  analyses  when  required, 
and  identifying  and  quality  ing  new  pa-t  types.  The  preferred  parts  list 
maintains  the  number  of  different  part  types  used  in  the  equipment  design 
to  a minimum.  This  maximizes  the  usage  of  preferred  puns  with  known 
reliability  characteristics.  The  parts  screering  and  vendor  control  tusks 
are  directed  at  controlling  and  maintaining  a high  dogrec  of  part  reliabil  - 
ity thi  our  fort  the  development  phase  of  a system..  Component  and  circuit 
stress,  ■ sts  and  analyses  are  performed  if  the  particular  operating  con- 
ditions of  u component  or  circuit  are  beyond  the  scope  of  a specification 
or  delate  the  established  reliability  of  a component  or  circuit.  The 
qualifying  of  new  part  types  when  it  is  necessary  to  use  a new  part  type  in 
the  design  of  a system  is  for  the  ass  ranee  of  the  Contractor  and  Engi- 
neering that  the  new  part  type  will  meet  the  reliability  requirements 
allocated  to  it. 

3)  Reliability  Evaluation  Phase  - the  reliability  tasks  that  describe  this 
phase  are  system  qualification  and  environmental  tests,  developmental 
tests,  failure  analyses,  and  the  system  reliability  demonstration  test. 

The  system  qualification  tests  arc  performed  to  determine  if  the  system 
meets  the  operational  objectives  of  the  contract  and  are  performed  under 
the  conditions  of  the  intended  field  use  of  the  system.  The  environmental 
fe°ts  arc  performed  to  remove  infant  mortality  failures  from  the  system 
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and  demonstrate  reliability  growth  during  in-house  system  testing.  The 
phenomenon  of  reliability  growth  is  detailed  in  ref.  (12).  During  the  sys- 
tem developmental  testing  some  failures  occur.  These  failures  are 
thoroughly  analyzed  to  determine  the  cause  of  failure  so  that,  if  necessary, 
design  fixes  can  be  implemented  early  in  the  development  testing  period. 

As  design  deficiencies  are  corrected  the  overall  reliability  of  the  system 
will  improve  with  time.  The  reliability  demonstration  tests  are  performed 
to  a MIL-SrD-781  environment  to  determine  the  achieved  system  reliabil- 
ity as  a re^-dt  of  all  of  the  reliability  nrogram  phase  contributions.  The 
primary  purpose  of  the  reliability  demonstration  test  is  to  damonstrate  to 
the  customer  that  the  system  meets  the  contractual  reliability  objectives. 
This  provides  incentive  to  the  contractor  to  properly  implement  the  relia- 
bility program  phases  as  failure  to  comply  to  the  contracted  reliability 
objectives  carries  severe  monetary  penalties. 

. 4. 2 Relevant  cost  efforts 

The  actual  costs  of  the  reliability  jgram  phases  (Cp,  Cp,  Cg)  are  those 
incurred  to  accomplish  the  tasks  u .lineuted  in  the  reliability  program  phase 
descriptions.  Labor  records  from  the  Systems  Effectiveness  Department, 
Radar,  Communications,  and  Support  departments  at  Ground  Systems  Group, 
and  the  Technology  Support  Department,  Data  Systems  and  Electro  Optical 
Division  at  Aerospace  Group  supplied  the  cost  data  for  the  data  base.  The 
costs  represent  only  those  expended  to  achieve  the  contractual  reliability. 

• 5 Data  Evaluation:  Determination  of  Data  Validity 

Throughout  the  data  acquisition  phase  of  the  study  the  data  sources  wore 
checked  to  verify  the  validity  of  the  data  collected  (c.  g. , the  existence  of  corn- 
patent  methods  of  cost  control  and  recording  in  the  form  of  task  effort  costs 
records).  To  make  a judgement  as  to  the  validity  of  the  actual  dita  itself  poses 
a unique  problem  of  definition.  Each  departmental  organization  is  responsible 
for  a specific  task  and  is  provided  funds  to  accomplish  the  task.  The  data  only 
reflects  the  fact  that  the  objectives  were  mot  and  what  costs  were  incurred  to 
accomplish  the  objectives.  What  the  data  does  not  reflect  is  how  the  tasks  were 
approached,  or  if  the  means  to  accomplish  the  tasks  were  optimized,  or  if 
problems  occurred  that  may  have  distracted  th8  direction  of  efforts  away  from 
the  objectives,  or  if  any  extensive  changes  were  implemented,  etc.  There- 
fore, the  validity  of  the  data  is  a subjective  determination,  especially  since  the 
dita  was  gathered  after  the  fact.  However,  because  all  of  the  data  is  internal 
to  the  Hughes  Aircraft  Company,  these  effects  are  certainly  smaller  than  they 
would  be  if  a number  of  companies  had  been  represented  in  the  data  base. 
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SECTION  3.0  DESCRIPTION  OF  DATA  ANALYSES 


3. 1 Data  Normalisation 


There  are  usually  a number  of  independent  variables  that  can  affect  a given 
dependent  variable.  In  this  study  we  will  particularly  be  investigating,  as 
dependent  variables,  reliability  and  cost.  The  important,  useful  independent 
variables  are  included  in  the  regression  models.  However,  there  are  several 
(independent)  variables  that  may  affect  the  dependent  variables  which  we  do 
not  particularly  want  to  study.  These  will  lx*  "normalised"  out  or  included  in 
the  regression  model.  These  variables  are: 

i)  Complexity 

ii)  Design  differences 

iii)  Environmental  differences 

iv)  Time  differences 

v)  Non-rclevant  coat  differences 

In  this  study  complexity  was  measured  by  parts  count.  The  variable  N (total 
parts  count)  and  (N*,  ND)  were  included  in  the  regression  models.  To  be 
"on  the  safe  side"  the  digital  parts  count  (Np)  on  each  system  wns  converted 
to  equivalent  analog  parts  count  and  a new  independent  variable,  NpA,  the 
total  number  cf  equivalent  analog  parts  was  included  in  the  regression  models. 
The  conversion  factor  used  was  a constant  2 x Np  =■  NA  (two  times  the  number 
of  digital  parts  equals  their  equivalent  number  of  analog  parts). 

The  design  effects  are  considered  to  be  negligible  because  all  ten  (10)  systems 
in  the  data  base  were  designed  by  the  Hughes  Aircraft  Company,  nine  (9)  of 
these  systems  were  designed  at  Hughes-Fullerton. 

The  environmental  differences  were  removed  from  the  MTBF's  by 
normalization.  This  normalization  was  important  only  when  predicted  or 
specified  MTBF  was  used  as  a dependent  variable.  Also,  when  ratios  (say 
®spec/®pred)  were  used  as  dependent  variables  the  normalization  factor  was 
nofneeden  because  it  cancelled  out.  The  basis  for  the  normalization  is  given 
in  Table  3. 1. 1.  The  k-factors  were  developed  using  the  fixed  ground  environ- 
ment as  the  normal  (k  factor  = 1)  case.  The  k-factors  are  composites  of  the 
various  environmental  factors  given  in  MII.-HDBK-217B  for  the  various  parts. 
The  weights  assigned  were  based  on  rough  estimates  of  part  distribution.  It 
turned  out  that  the  shipboard  and  ground  mobile  factors  wore  identical. 

As  can  be  seen  from  Table  2.3. 1 the  ten  (10)  systems  forming  the  data  base 
are  of  relatively  recent  vintage  so  that  time  effects  are  considered  negligible. 
Finally,  to  remove  non-relevant  cost  differences  (e.g. , the  changing  value  of 
the  dollar)  all  costs  are  measured  in  man-days. 
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TABLE  3. 1. 1.  k- Factors  for  Environmental  Normal  l rat  ion 


adjusted 
( 0x  k -factor) 

Sys.  # 

Environment 

e' 

spec 

e’  . 

prod 

k-fsetor 

*tpec 

%red 

1 

Shipboard 

90 

98 

360 

392 

2 

Shipboard 

125 

146 

4 

500 

584 

3 

Shipboard 

500 

574 

4 

2000 

2296 

4 

Shipboard 

290 

270 

4 

1160 

1080 

5 

Shipboard 

182 

165 

1 

4 

728 

660 

6 

Ground, 

Fixed 

190 

210 

1 

190 

210 

7 

Ground, 

Fixed 

250 

216 

1 

250 

216 

8 

Ground, 

Fixed 

4000 

5721 

1 

4000 

5721 

9 

Ground, 

Mobile 

184 

369 

4 

736 

1476 

10 

Ground, 

Mobile 

200 

217.5 

4 

800 

868 

3. 2 The  Regression  Variables 

The  various  seta  of  dependent  and  independent  variables,  one  hundred  twenty 
four  (124)  in  all,  are  shown  in  Table  3.2.1.  Each  set  was  rui)  on  each  of  the 
five  model  types  discusseJ  in  the  next  section  with  the  exception  that  in  a very 
few  cases  where  there  existed  a large  number  of  independent  variables 
(e.g.  3)  and  a small  number  of  data  sets  (as  in  the  gain  analysis)  the  second 
degree  model  with  cross-product  terms  could  not  be  run  because  the  degrees - 
of -freedom  were  too  small.  Not  all  of  the  independent  variable  sets  provided 
good  predictions  for  the  various  dependent  variables  so  the  list  in  Table  3. 2. 1 
gives  all  those  sets  tried,  not  the  sets  that  were  good  fits. 

For  the  1-  numbered  seta  the  dependent  variable  is  always  cost.  The  inde- 
pendent variable (s)  are  those  that  would  normally  be  available,  at  least  in 
estimated  form,  early  enough  to  be  of  use  in  predicting  costs. 
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Selection  No 


spec  pred 

e 

spec 

®pred 

N 
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TABLE 

Selection  No  [ 

1*24 

1-25 

1-26 

1-27 

1-28 

1-29 

i-30 

1-31 

1-32 

1-33 

1-34 

1-35 

1-36 

1-37 

1-38 

1-39 

1-40 

1-41 

1-42 

1-43 

1-44 

1-45 

1-46 


TABLE  3.2.1.  The  Regression  Variables  (Continued) 


Selection  No 

Dependent  Variable 

. ...  . 1 

Independent  Variable  (s) 

1-47 

CP 

^spec’  ^EA 

1-48 

CP 

6spec’  NA’  ND 

1-49 

CP 

®pred*  N 

1-50 

cp 

®pred’  ^EA 

1-51 

CP 

1 

0pred‘  NA’  ND 

1-52 

CE 

^spec^pred*  ^ 

1-53 

CE 

^spec^pred’  NEA 

1-54 

CE 

®spec^^pred*  NA’ 

1-55 

CE 

0 , N 

spec* 

1-56 

CE 

9spec*  NEA 

1-57 

CE 

0spec*  NA*  ND 

1-58 

CE 

®nred’  ^ 

1-59 

CE 

0pred’  NEA 

1-60 

CE 

j 

®pred*  NA*  ND 

2-1 

SE 

CT 

2-2 

0E 

N 

2-3 

6E 

nea 

2-4 

6E 

na-  nd 

2-G 

SE 

CT,  N 

2-6 

eE 

C N 

T’  EA 

2-7 

SE 

CT*  NA*  ND 

2-8 

GE 

CE 

2-9 

0E 

CE*  N 

TABLE  3.2.1.  The  Regression  Variables  (Continued) 


Selection  No 

Dependent  Variable 

Independent  Variable  (s) 

3-11 

ei 

N,  CT 

3-12 

9I 

nea*  ct 

3-13 

©T 

Na,  nd,  Ct 

3-14 

0D 

N.  C.f 

3-15 

®D 

nea*  ct 

3-16 

°D 

na*  nd*  ct 

3-17 

0D 

N.  CD 

3-18 

9d 

nea*  cd 

3-19 

eD 

na-  ND’  cd 

3-20 

6P 

N.  CT 

3-21 

eF 

NEA’  ct 

3-22 

GP 

na*  nd*  ct 

3-23 

GP 

N.  Cp 

3-24 

®P 

nea*  cp 

3-25 

®p 

na*  nd*  cp 

3-26 

e 

N,  CT 

-27 

eE 

NEA’  ct 

3-28 

0E 

na*  nd*  ct 

3-29 

SE 

N,  CE 

3-30 

6E 

nea*  CE 

3-31 

0E 

nA’  nd*  ce 

3-32 

0D 

3-33 

9P 

Cp 
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TABLE  3.2.1.  The  Regression  Variables  (Continued) 


Selection  No 

Dependent  Variable 

Independent  Variable (s) 

3-34 

0E 

CE 

3-35 

gd 

Cd 

3-3€ 

GP 

Cp 

3-37 

°E 

CL 

3-38 

gt 

CT,  N 

3-39 

G,j. 

CT*  NEA 

3-40 

G1 

CT*  N.V  ND 

3-41 

gt 

c c c 

'■'D’  P’  E 

3-42 

gt 

c*  c1  c' 

'■'D*  P*  E 

The  2-  numbered  sets  are  those  with  the  final,  achieved  MTBF,  0jr,  as  the 
dependent  variable.  The  natural  independent  variables  are  cost  and  parts 
count  and  these  are  the  variables  that  are  used. 

The  3-  numbered  sets  form  the  foundation  for  the  reliability  gain  and  optimal 
allocation  analysis.  The  dependent  variables  are,  in  turn,  phaue^  MTBF’s,  ( 
phase  gains,  and  total  gain.  Among  the  independent  variables,  Cj),  Cp  and  Cjr 
require  special  mention.  These  are  the  relative  costs  of  the  three  periods 
and  are  useful  in  the  allocation  problem.  For  exapple,  consider  3-42,  For 
a fixed  total  gain  Gp.  the  variables  Cp,  Cp  and  Cg  can  be  regressed  on  G? 
and  the  coefficients  estimated.  This  regression  equation  can  then  be  used  in 
attacking  the  optimal  allocation  problem. 

3. 3 Description  of  the  Models  and  Measures  of  Fit 

Each  of  the  following  five  (5)  models  was  fitted  to  each  of  the  one  hundred 
twenty  four  (124)  variable  sets  described  in  the  previous  section.  It  was  not 
expected,  nor  even  desired,  that  all  models  would  fit  all  the  variable  sets. 

Nor  was  it  desired  that  there  be  at  least  one  good  model  fit  on  every  variable 
set.  All  that  is  needed  is  that  there  be  at  least  one  good  model  for  each 
dependent  variable  of  interest.  For  example,  in  Table  3.2.1,  the  1-  numbered 
variable  sets  include  fifteen  (15)  with  CT  as  the  dependent  variable.  As  a 


worst  case  we  only  need  one  model  to  be  a good  fit  to  one  of  these  fifteen 
variable  sets  with  C j as  the  dependent  variable.  The  five  models  used  are 
(<  represents  the  random  error  term): 

Linear 


Y = aft  + a.X.  + ...  + a X + c 
oil  n n 


Ln— Linear 


In  Y = In  aQ  + ax  In  Xx  + . . . + an  ln  xn  + e 


Exponential 


Y = e 


(a. 


alXl 


a X 
n n 


+ €) 


Second  Degree 


Y * a0  + allXl  + + 


+ amX»  + an2Xn  + £ 


Second  Degree  With  Cross  Products  (example,  for  n = 2) 

Y ‘ a0  f allXl  + + “2 1*2  + * a(12)XlX2  + £ 


The  measures  of  fit  for  these  five  models  are  R and  R.  E..  These  measures 
are  described  informally  in  section  1.2  and  formally  in  section  1.3. 

3. 4 Description  of  Model  Estimation  Programs 

3. 4. 1 General  description 

The  reliability  acquisition  cost  study  software  routines  utilized  for  the  model 
estimation  analysis  of  the  study  data  base  were  constructed  utilizing  "canned 
routines,"  i.e.,  programs  existing  in  Hughes  software  library,  and  new  soft- 
ware routines  were  used  to  merge  and  sort  the  outputs  of  the  "canned  routines" 
into  condensed  data  sets.  The  routines  for  each  model  examined  were  the 
same  except  for  the  input  data  formatting.  All  software  was  programmed  in 
FORTRAN  IV  utilizing  the  Hughes  computer  installation  consisting  of  an 
IBM  370-165  with  extensive  library  modules. 
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3. 4. 2 IBM  routines 

The  "canned  routines"  used  are  from  the  IBM  Scientific  Subroutine  Package 
for  a multiple  linear  regression  program.  The  multiple  linear  regression 
program  consists  of  a main  routine  named  REGRE,  a special  user  supplied 
input  subroutine  named  DATA,  and  four  other  subroutines  from  the  Scientific 
Subroutine  package:  CORRE,  ORDER,  MINV,  and  MULTR.  With  the  excep- 
tion of  the  main  routine  REGRE,  all  of  the  subroutines  were  available  in 
load  module  form  only  and  could  not  be  accessed.  However,  the  formatting 
of  the  main  routine  REGRE  could  be  accessed  enabling  the  output  data  from 
the  multiple  linear  regression  program,  in  this  case  the  intercept  term,  the 
regression  coefflcient(s),  and  the  multiple  correlation  coefficient  for  each 
model  selection  to  be  written  into  a new  data  set. 

3.4.3  Multiple  linear  regression  methodology 

Given  a linear  equation  in  two  variables,  Y = a + pX  where  a is  the  Y intercept 
and  P is  the  slope  of  the  line,  the  problem  of  finding  the  "best  fit"  line  to  a 
given  set  of  N points  (xi.yi),  (x2,y2)»  • • • . (Xm.Yn)  is  *°  determine  the  values 
a and  b so  that  the  sum  of  the  squares  of  the  dlnerence  between  the  estimated 
values  of  Y (given  by  Y = a + bX)  and  the  observed  values  of  Y is  a minimum. 
This  is  the  least  squares  approach. 

The  constants  a and  b of  the  equation  $ = a + bX  are  solutions  of  two  linear 
equations  called  normal  equations . 


N 


aN  + b. 


i=l 


(3.4.1) 


N 


ExiYi  • 

i=l 


(3.4.2) 


The  constants  a and  b are  given  by 


N N N 

"X>tYi  - ExEyi 


i=l  i=l  i=l 


N 

/ N \ 

2 

NI>f  - 

Ex 

| 

i=l 

(3. 4. 3) 
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and 


a = Y - b X 


I 

l 

i 

» 

\ 

i 

(3.4.4) 


where  X Is  the  mean  of  the  X-values  and  Y is  the  mean  of  the  Y-values. 

In  the  multivariable  case  the  normal  equations  are  similar  to  the  linear  case. 
The  Y-values  are  the  dependent  variables,  i.e. , ir>  the  analysis  of  the  data 
the  dependent  variables  can  be  reliability  phase  cost  or  total  cost,  system 
MTBF's,  or  reliability  gain,  and  the  X-values  are  the  independent  variables, 
i.e.,  number  of  system  parts,  reliability  phase  costs,  system  MTBF's,  or 
reliability  gain,  depending  on  the  relationship  that  is  being  analyzed.  The 
relationships  of  dependent  and  independent  variable  (s)  are  defined  by  the 
selection  numbers  in  the  Regression  Variables  Table,  Table  3.2.1. 

3.4.4  Data  sets 

The  data  sets  were  created  from  the  study  data  base  according  to  model  type. 
The  data  sets  were  input  to  the  appropriate  model  routine  and  the  outputs  from 
each  model  routine  were  merged  into  one  final  data  set  which  was  sorted  for 
best  fit  parameters  by  model.  The  model  estimation  procedure  is  shown  in 
Diagram  1. 

3.4.5  Data  sort 

New  software  was  created  to  calculate  the  deviation  (error)  from  the  regres- 
sion analysis  data.  The  measures  of  "goodness  of  fit"  (R.  E.  and  R,  defined  in 
Section  1. 3)  were  determined  for  each  selection  for  all  models  and  sorted  for 
the  "best  fit"  by  selection  number. 


SECTION  4.0  RESULTS  OF  DATA  ANALYSES 


4. 1 The  Prediction  Equations  for  Reliability  Coats 

'Hie  variable  sets  numbered  1-,  aa  previously  mentioned  in  Section  3.2,  all 
have  reliability  coat*  In  aome  form  or  other,  aa  the  dependent  variable.  The 
distribution  of  the  sixty  (60)  seta  (selections)  la 


Selection  Numbers 
1-1  through  1-8 
1-25  through  1-33 
1-7  through  1-12 
1-34  through  1-42 
1-13  through  1-18 
1-43  through  1-51 
1-19  through  1-24 
1-52  through  1-60 


Dependent  Variable 


The  results  of  the  model  fits  are  given,  by  model,  in  Table  4. 1. 1.  In  the 
following  sections  we  discuss  the  best  fit  results  for  each  individual  oost  cate- 
gory and  total  costs. 


4.1.1  Results  for  total  cost,  CT 

The  results  of  the  best  fit  for  Cj  as  a dependent  variable  are  given  in 
Table  4. 1. 2.  It  is  clear  from  this  table  and  Table  3. 2. 1 that  no  one  independ- 
ent variable  is  a good  predictor  of  Cj.  Far  and  away  the  best  fits  are  the 
2nd  degree  cross  products  model  with  the  independent  variables  ©spec.  NA* 
No  (1-30)  and  0pre(j,  NA»  ND  U -33).  These  equations  are  given  below. 


C„  * -24,814  + 41.30  0 -O.OO8102  + 1. 89N.  - 0.000015N2 

T spec  spec  A A 

«•«<»«„  * 0, 0000082 - 0. 0013^6^  - 0.00321^9^  . (!) 


r = -28,154  * 24.856  . - 0.00J262  . * 2.34N.  - 0.  000027  N, 

T pred  pred  A A 

+ 0.76Nd  - 0.000061N^  - 0. 0011NA6pred  - 0. 00064ND©pred  . (2) 
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TABLE  4. 1. 1 Goodness  of  Pit  Results  for  Rellsbillty  Cost 


Linear  Ln -Linear  Expon  2nd  Deg  2nd  Deg  CP 


lection 

R.E. 

I 

R.E. 

R.E. 

R.E. 

1 

R.E. 

5 

1-1 

1.12 

56.27 

1.17 

46.93 

1.19 

48.06 

0.75 

35.04 

0.75 

35.04 

1-2 

1.08 

60.13 

1.09 

51.39 

1.18 

52.92 

1.01 

61.54 

1.01 

61.54 

1-3 

1.07 

59.44 

1.07 

51.63 

1.20 

56.42 

0.98 

58.23 

0.98 

58.23 

1-4 

0.88 

47.54 

0.95 

43.61 

0.95 

40.93 

0.81 

45.97 

0, 81 

45.97 

1-5 

0.87 

48.22 

0.95 

43.35 

0.93 

41.07 

0.84 

47.46 

0.84 

47.46 

1-6 

0.86 

48. 80 

0.93 

42.26 

0.92 

41.21 

0.75 

44.60 

0.61 

37.68 

1-7 

1.09 

19.31 

1.09 

19.74 

1.87 

27.64 

0.98 

19. 73 

0.98 

19.73 

1-8 

0.76 

15. 51 

1.03 

19.47 

6.21 

45. 42 

0.69 

12.43 

0.69 

12.43 

1-9 

0, 81 

15.07 

1.09 

19.38 

5.01 

30. 28 

0.62 

12.99 

0.62 

12.99 

1-10 

0.94 

16.08 

1.05 

15.92 

0.95 

15.25 

0.85 

15.27 

0. 85 

15.27 

1-11 

0.97 

16.86 

1.07 

16.48 

0.98 

16.23 

0. 88 

16.40 

0..88 

16.40 

1-12 

0.90 

15.31 

0.61 

12.84 

0.92 

14.91 

0.76 

13. 84 

0.65 

10.49 

1-13 

1.10 

60.27 

1.17 

53.80 

1.20 

55.33 

0.67 

40.02 

0.67 

40. 02 

1-14 

1.11 

64. 26 

1.17 

59.26 

1.22 

56.68 

1.05 

64.  85 

1.05 

64.85 

1-15 

1.10 

64.68 

1.13 

59.47 

1.19 

58.38 

1.02 

62.70 

1.02 

62.70 

1-16 

0.73 

49.06 

0.87 

49.95 

0. 80 

44.78 

0.72 

49.26 

0.72 

49.26 

1-17 

0.72 

49.31 

0.87 

49.34 

0.77 

43. 82 

0. 72 

49.24 

0.72 

49.24 

1-18 

0.72 

49.  53 

0.86 

49.25 

0.78 

44. 22 

0.63 

43. 35 

0.41 

32.60 

1-19 

1.12 

82.39 

1.20 

59.16 

1.18 

61.43 

0.88 

55.40 

0.88 

55. 40 

1-20 

l."5 

92.76 

1.10 

62.94 

1.19 

69.12 

0.98 

100.00 

0.98 

100. 0( 

1-21 

1.05 

90.83 

1.11 

62.65 

1.21 

73.22 

0.97 

99.12 

0.97 

99.12 
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TABLE  4. 1. 1 Goodneaa  of  Fit  ResuUa  for  Reliability  Coat  (Cont) 


Linear 

Ln-Linear 

Expon 

2nd  Deg 

2nd  Deg  CP 

Selection 

R.  E. 

E 

R.E. 

E 

R.E. 

E 

R.K. 

_B_ 

R.E. 

R 

1-22 

1.02 

73.05 

1.08 

54. 48 

1.11 

55.63 

0.90 

87.73 

0.90 

G7.73 

1-23 

1.01 

73.14 

1.08 

53. 87 

1.09 

55.80 

0.93 

86.93 

0.93 

86.93 

1-24 

0.99 

74.18 

1.06 

52.33 

1.08 

55.36 

0.83 

91.90 

0.76 

76.93 

1-29 

0.85 

47.73 

0.86 

41.80 

1.33 

74.26 

0.50 

31.48 

0.45 

30.76 

1-26 

0.84 

48.47 

0.86 

41.38 

1.09 

60.97 

0.50 

31.26 

0.49 

30.61 

1-27 

0.84 

48.59 

0.83 

41.36 

1.14 

64.49 

0. 38 

27.84 

0.11 

15. 51 

1-28 

0.88 

42.38 

1.04 

38.24 

1.03 

33.03 

0.69 

47.76 

0.46 

37.51 

1-29 

0.86 

41.73 

1.04 

37.68 

0.99 

32. 88 

0.74 

48.02 

0.33 

32.92 

1-30 

0. 85 

41.92 

1.02 

36.22 

0.98 

33.16 

0.61 

45.44 

0.00 

1.15 

1-31 

0. 88 

42.73 

1.06 

35.52 

1.01 

34.37 

0.71 

44.46 

0.21 

17.85 

1-32 

0.86 

42. 51 

1.05 

34.71 

0.98 

34. 70 

0.77 

43.56 

0.15 

20.59 

1-33 

0.85 

43.01 

1.03 

32.76 

0.97 

34.88 

0.62 

42.  88 

0.01 

6.05 

1-34 

0.71 

16.04 

0.54 

14. 16 

14.55 

79.62 

0.31 

10.83 

0.31 

10.77 

1-35 

0.78 

16.68 

0.62 

14.99 

9.99 

64.78 

0.36 

11.95 

0.36 

11.93 

1-36 

0.60 

14.69 

0.11 

6.23 

28.67 

100.00 

0. 16 

6.60 

0.15 

6.75 

1-37 

0.75 

15.32 

1.02 

17.91 

4.08 

36.57 

0.24 

8.43 

0.24 

8.24 

1-38 

0.76 

15.81 

1.03 

18.51 

4.92 

39.57 

0.24 

8.18 

0.24 

8.31 

1-39 

0.66 

13.90 

0.45 

12.48 

5. 67 

43. 24 

0. 24 

8.75 

0.11 

5.86 

1-40 

0.80 

14.96 

1.05 

16.77 

3.44 

32. 68 

0.18 

6.75 

0.15 

6.00 

1-41 

0.81 

15.11 

1.07 

17.69 

4.11 

35. 59 

0.19 

7.19 

0.19 

7.15 

1-42 

0.73 

14.24 

0. 54 

13.20 

4.22 

36.08 

0.22 

7.59 

0.11 

5.14 

27 


J 


i 


J 


TABLE  4.1*1  Qoodneaa  of  Fit  Result*  for  Reliability  Cost  (Cont) 


Linear 

Ln  -Linear 

Expon 

2nd  Deg 

2nd  Deg  CP 

Selection 

R.E. 

R 

R.E. 

1 

R.E. 

R 

R.E. 

R 

R.E. 

R 

1-43 

0.70 

48.97 

0.81 

49.18 

0.76 

45.19 

0.46 

32.93 

0.46 

32. 02 

1-44 

0. 70 

49.75 

0.81 

48.94 

0.44 

44.68 

0.49 

33.20 

0.49 

33. 16 

1-45 

0. 70 

49. 51 

0.79 

47.87 

0.74 

44.  89 

0.27 

27.95 

0.09 

11.65 

1-46 

0.64 

34.23 

0.96 

42.27 

0.88 

31.38 

0.53 

41.08 

0.37 

33. 75 

1-47 

0.61 

33.94 

0.95 

41.99 

0.77 

30.01 

0.57 

38.75 

0.35 

30.48 

1-48 

0.60 

34.45 

0.92 

41.14 

0.77 

30.33 

0.40 

34.45 

0.03 

9.39 

1-49 

0.64 

36.28 

0.93 

40.41 

0.86 

33.15 

0. 51 

36. 70 

0.16 

18.31 

1-50 

0.62 

36.75 

0.93 

39.95 

0.77 

32.75 

0.56 

34.76 

0.20 

20.70 

1-51 

0.62 

37.09 

0.90 

38.70 

0. 79 

32.54 

0.38 

31.66 

0.01 

3.73 

1-52 

0.9  'j 

71.79 

1.00 

54.93 

2.83 

100.00 

0.58 

53.  80 

0.48 

56.85 

1-53 

0.98 

71.56 

0.99 

54.52 

2.39 

100.00 

0.50 

47.11 

0.  46 

56.  26 

1-54 

0.98 

72.37 

0.97 

52.24 

2.02 

100.00 

0.49 

45. 99 

0.11 

28.  70 

1-55 

1.01 

82.37 

1.12 

50.69 

1.14 

52. 63 

0.79 

100.00 

0.54 

73.78 

1-56 

1.00 

81.02 

1.12 

49.54 

1.13 

52.30 

0.  82 

100.00 

0.30 

64.67 

1-57 

0.99 

80.43 

1.10 

47.45 

1.12 

51.77 

0.73 

100.00 

0.01 

13.69 

1-58 

1.01 

79.  87 

1.15 

46.30 

1.13 

53.68 

0.  85 

95. 76 

0.31 

41.05 

1-59 

1.00 

78.79 

1.15 

44.67 

1.12 

53.57 

0.88 

96.79 

0.15 

41.15 

1-60 

0.99 

79.09 

1.13 

43.53 

1.11 

53.16 

0.78 

94.26 

0.06 

27.96 

TABLE  4.1.2  Beat  Fit  Results  for  CT 


Selection  No. 

Model 

R.E. 

1. 

Model 

R_ 

1-1 

2NDD 

0.75 

2NDD 

35.04 

1-2 

2NDD 

1.01 

LNL 

51.39 

1-3 

2NDD 

0.98 

LNL 

51.63 

1-4 

2NDD 

0.81 

EXP 

40.93 

1-5 

2NDD 

0.84 

EXP 

41.07 

1-6 

2DCP 

0.61 

2DCP 

37.68 

1-25 

2DCP 

0.45 

2DCP 

30.76 

1-26 

2DCP 

0. ’9 

2DCP 

30.61 

1-27 

2DCP 

0.11 

2DCP 

15.51 

1-28 

2DCP 

0.46 

EXP 

33.03 

1-29 

2DCP 

0.33 

EXP 

32.88 

1-30 

2DCP 

0.00 

2DCP 

1.15 

1-31 

2DCP 

0.21 

2DCP 

17.85 

1-32 

2DCP 

0.15 

2DCP 

20.59 

1-33 

2DCP 

0.01 

2DCP 

6.05 
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4. 1.2  Results  for  design  cost, 

The  best  fit  results  for  Cp  given  in  Table  4. 1.3  indicate  that  virtually  all  the 
independent  variables  are  good  predictors  of  Cq.  Furthermore,  inspection  of 
Table  4. 1. 1 indicates  that  all  of  the  models  do  a reasonably  good  job  of  fitting. 
Since,  typically,  a user  will  have  various  independent  variables  available  at 
various  times  the  results  for  the  best  fits  are  given  in  Table  4. 1.4.  A blank 
entry  in  the  table  indicates  that  the  particular  independent  variable  (row)  is  not 
used  on  that  particular  selection  number  (column).  The  mathematical  form  of 
the  models  may  be  found  either  in  Section  1.2  or  Section  3.3. 

4. 1.3  Results  for  part  cost,  Cp 

As  can  bs  .seen  in  Table  4. 1. 5 (and  using  Table  3.2. 1)  no  one  independent  vari- 
able does  well  as  a predictor  for  Cp.  However,  again,  the  sets  of  independent 
variables  0gpec.  NA*  ND  (1-48)  and  ©pred*  Na*  Nd  (1-51)  are  good  predictors 
of  Cp.  These  equations  are  given  below. 

C_,  = -9,133  + 15.020  - O.OO2902  + 1.10N.  - O.OOOOION^ 

P spec  spec  A A 

- 1.07N„  + 0.000049Ni  - O.OOO63N.0  ~ 0. OOO44N-0  M (3) 
D DA  spec  D spec  ' 7 

C„  = -11,603  + 10.080  . - O.OO1302  . + 0.92N.  - 0.000011N2 

P pred  pred  A A 

+ 0.  52Nd  - 0. 00002IN^  - 0. 00031^  ©pred  - 0.  OOO61ND0pred  (4) 

4. 1.4  Results  for  evaluation  cost,  Cjr 

None  of  the  single  (independent)  variable  cases  provided  suitable  fits  as  can  be 
seen  by  using  Table  3.2. 1 and  Table  4. 1.6.  Again,  the  independent  variables 
©spec*  Na.  Nd  and  ©pred*  Na,  Nd  provide  satisfaictory  fits  for  the  2nd  degree 
cross  products  model.*  These  equations  (selections  1-57  and  1-60)  are  given 
below. 


C=  -17,137  + 28.170  - 9.OO5602  + 0.79N.  - 0.0Q00048N^ 

E spec  spec  A A 

+ 1 . 78N^  - 0. 000051N?.  - 0. 00067N  A 0 - 0. 0031^0  (5) 

D D A spec  D spec  ' 7 

C„  - -18,027  + 15.,  770  , - O.OO2102  . + 1.49N.  - 0.000017N^ 

E pred  pred  A A 

+ 0.41N„  - 0.000048N?,  - O.OOO8ON.0  . -O.OOO14N„0  . . (6) 

D DA  pred  D pred  ' 7 
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TABLE  4. 1.3  Best  Fit  Results  for  CD 


Selection  No. 

Model 

R.  E. 

Model 

R 

1-7 

2NDD 

0.98 

UN 

19.31 

1-8 

2NDD 

0.69 

2NDD 

12.43 

1-9 

2NDD 

0.62 

2NDP 

12.99 

1-10 

2NDD 

0. 85 

EXP 

15.25 

1-11 

2NDD 

0.88 

EXP 

16.23 

1-12 

LNL 

0.61 

2DCP 

10.49 

1-34 

2DCP 

0.31 

2DCP 

10.77 

1-35 

2DCP 

0.36 

2DCP 

11.93 

1-36 

LNL 

0.11 

LNL 

6.23 

1-37 

2DCP 

0.24 

2DCP 

8.24 

1-38 

2DCP 

0.24 

2NDD 

8.18 

1-39 

2DCP 

0.11 

2DCP 

5.  86 

1-40 

2DCP 

0.15 

2DCP 

6.00 

1-41 

2DCP 

0.19 

2DCP 

7.15 

1-42 

2DCP 

0.11 

2DCP 

5.14 
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TABLE  4. 1. 4 Model  Coefficients  for  Design  Cost 


3.  Starting  with  the  general  form  of  the  model  given  on  page  21,  substitute 
into  the  model  the  intercept  (a0),  the  coefficients  from  table  t 1.4,  and 
values  of  the  independent  variables  to  find  the  value  of  Cp. 


TABLE  4. 1. 5 Beat  Fit  Results  for  Cp 


Selection  No. 

Model 

R.E. 

Model 

R 

1-13 

2NDD 

0.67 

2NDD 

40.02 

1-14 

2NDD 

1.05 

EXP 

56.68 

1-15 

2NDD 

1.02 

EXP 

58.38 

1-16 

2NDD 

0.72 

EXP 

44.78 

1-17 

2NDD 

0.72 

EXP 

43.82 

1-18 

2DCP 

0.41 

2DCP 

32.60 

1-43 

2DCP 

0.46 

2DCP 

32.62 

1-44 

EXP 

0.44 

2DCP 

33.16 

1-45 

2DCP 

0.09 

2DCP 

11.65 

1-46 

2DCP 

0.37 

EXP 

31.38 

1-47 

2DCP 

0.35 

EXP 

30.01 

1-48 

2DCP 

0.33 

2DCP 

9.39 

1-49 

2DCP 

0.16 

2DCP 

18.31 

1-50 

2DCP 

0.20 

2DCF 

20.70 

1-51 

2DCP 

0.00 

2DCP 

3.73 

TABLE  4.1.6  Best  Fit  Results  for  Cr, 

E 


Selection  No. 

Model 

R.E. 

Model 

R. 

1-19 

2NBD 

0. 88 

2NDD 

55.40 

1-20 

2NDD 

0.98 

LNL 

62.94 

1-21 

2NDD 

0.97 

LNL 

62.65 

1-22 

2NDD 

0.90 

LNL 

54. 48 

1-23 

2NDD 

0.93 

LNL 

53.87 

1-24 

2DCP 

0.76 

LNL 

52.33 

1-52 

2DCP 

0.48 

2NDD 

53.80 

1-53 

2DCP 

0.4  C 

2NDD 

47.11 

1-54 

2DCP 

0.11 

2DCP 

26.70 

1-55 

2DCP 

0.54 

LNL 

50.69 

1-56 

2DCP 

0.30 

LNL 

49.  54 

1-57 

2DCP 

0.01 

2DCP 

13.  69 

x-58 

2DCP 

0.31 

2DCP 

41.05 

1-59 

2DCP 

0.15 

2'JCP 

41.15 

1-60 

2DCP 

0.06 

2DCP 

27.96 
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Examples  (Hie  systems  used  in  the  examples  were  randomly  selected) 
1)  Predicting  total  cost  C?: 

System  No.  6 has 


6 

spec 

0 , 
pred 


- 190 
= 210 
= 28,429 
= 8,064 
= 16,699. 


Using  equation  (1)  we  obtain 


CT  = -24,814  + 41.30(190)  - 0.0081(190)2  + 1.89(28,429) 

- 0. 000015(28, 429) 2 + 0.40(8,064)  - 0. 0000082(8, 064)2 

- 0.  0013(190)(28, 429)  - 0. 0032(190) (8, 064) 

= 16,466.53 


which  is  in  close  agreement  with  CT(OBS)  = 16,699.  Also,  equation  (2)  leads  to 
CT  = -28,154  + 24.85(210)  - 0.0032(210)2  + 2.34(28,429) 

- 0. 000027(28, 429)2  + 0.76(8,064)  - 0.000061(8, 064) 2 

- 0.0011(210)(28,429)  - 0.00064(210)(8,064) 

= 16,137.65. 


2)  Predicting  design  cost  C^: 
System  No.  1 has 


0 

spec 


360 

392 

28,097 
14, 804 
214  • 
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To  avoid  repetition  we  will  use  just  the  set  08pec,  na»  nd*  From 
Table  4.1.4  (1-39) 

CD  = 195.24  + 0.11(360)  - 0. 000032(3G0)2  + 0.023(28,097)  - 0.00000015 

x (28, 097) 2 - 0.094(14,804)  + 0. 0000035(14, 804)2  - 0.000018(360) 
x 28,097  + 0.000048(360)  (14, 804) 

= 207.71. 


This  agrees  very  well  with  the  Cq(OBS)  = 214. 
3)  Predicting  part  cost  Cp: 

System  No.  3 has 


e 

spec 

e ^ 
pred 


N 


A 


N 


D 


Equation  (3)  gives 


2,000 
2,296 
9,461 
1,852 
4,093  . 


Cp  = -9,133  + 15.02(2,000)  - 0. 0029(2,  OOO)2  + 1.10(9,461) 

- 0. 000010(9, 461)2  - 1.07(1,852)  - 0. 000049(1, 852)2 

- 0.00063(2, 000) (9, 461)  - 0. 00044(2, 000)(1, 852) 

= 3,454.80* 


This  result  agrees  well  with  Cp(OBS)  = 4,093. 
4)  Predicting  evaluation  cost  Cg  : 

System  No.  4 has 


0 

spec 

®pred 


1,160 

1,080 
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Na  - 19,567 

Nd  = 2,116 

CE  = 2,464. 

Using  Equation  (5) 

C£  = -17,137  + 28.17(1,160)  - 0. 0056(1, 160)2  + 0.79(19,567) 

- 0. 0000048  x (19, 567)2  + 1.78(2,116)  - 0. 000051(2, 116)2 

- 0.00067(1, 160)(19, 567)  - 0.0031  x (1,160)(2. 116) 

= 2,346.55. 


The  Prediction  Equations  for  Final  Mr"BF  ©e 

The  "final"  MTBF,  ©g,  is  a fundamental  part  of  the  gain  analysis  to  be  treated 
in  Section  4. 3.  The  calculation  of  ©e  is  discussed  in  detail  in  that  section. 
Here,  it  is  sufficient  to  present  the  ideas  in  words.  The  final  MTBF,  0g, 
refers  to  the  mean  time  between  failures  achieved  as  a result  of  the  develop- 
mental program.  The  time  span  used  to  compute  ©g  includes  the  demonstra- 
on  test  time  even  though,  typically,  the  demonstration  test  is  relatively 
(relative  to  the  developmental  testing  process)  short.  Also,  ©e  must  be  com- 
puted with  extreme  caution.  For  example,  it  is  not  sufficient  to  compute  6g 
as  the  total  test  time  (developmental  and  demonstration)  divided  by  the  total 
number  of  failures  experienced  in  this  time.  This  calculation  will  bias  ©g  to 
the  low  side  because  many  of  the  failures  are  nonrecurring  (e.  g. , design 
correctable)  and  should  not  be  counted  in  a rnaHwtin  assessment  of  an  achieved 
MTBF.  Thus  the©g's  used  were  computed  from  the  equation  for  the  instanta- 
neous MTBF  given  in  Section  4.3. 

The  be  '*  results  are  given  in  Table  4.2. 1.  Selections  Nos.  2-1  through 
..  il  ► g as  the  dependent  variable  while  2-12  through  2-22  have©E, 
normalized  for  environment,  as  the  dependent  variable.  We  will  use  only 
equations  from  the  sets  2-1  through  2-11.  Although  good  fits  are  obtained  in 
2-12  through  2-22  it  is  not  hard  to  understand  why  the  fits  are  not  as  good  as 
the  2-1  through  2-11  sets:  although  the  various  systems  had  different  use 
environm  most  of  the  developmental  and  demonstration  test  time  was 
under  id'  .al  conditions  for  all  the  systems.  Thus,  the  normalization  of 
©E  is  sk..piy  not  proper. 
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TABLE  4. 2. 1 Best  Fit  Results  (or  Final  MTBF,  0£ 


Selection 

Model 

R«JEi 

Model 

1 

2-1 

2NDD 

0. 13 

2NDD 

4.37 

2-2 

2NDD 

0.30 

2NRD 

7.19 

2-3 

2NDD 

1.15 

2NDD 

12. 12 

2-4 

LIN 

1.11 

LNL 

10.57 

2-5 

UN 

0.01 

LNL 

1.26 

2-6 

UN 

0.00 

UN 

0.87 

2-7 

UN 

0.00 

UN 

0.89 

2-8 

LNL 

0.11 

LNL 

3.88 

2-9 

LNL 

O.Oy 

LNL 

3.62 

2-10 

LNL 

0.08 

LNL 

3.38 

2-11 

LNL 

0.00 

LNL 

0.81 

2-12 

2NDD 

0.06 

2NDD 

4.05 

2-13 

2NDD 

0.76 

2NDD 

26.67 

2-14 

2NDD 

1.13 

2NDD 

33.  58 

2-15 

LNL 

1. 24 

LNL 

36.24 

2-16 

LIN 

0. 27 

UN 

10.58 

2-17 

UN 

0.29 

UN 

11.41 

2-18 

UN 

0.20 

UN 

11.03 

2-19 

UN 

0. 10 

UN 

5.52 

2-20 

UN 

0.10 

UN 

5.26 

2-21 

UN 

0.09 

UN 

5.33 

2-22 

UN 

0.07 

UN 

5.80 
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We  have  selected  the  following  seta: 


Selection  No. 

Model 

independent  Variable 

2-5 

ln-linear 

ct,n 

2-7 

ln-llnear 

ct*na*nd 

2-9 

ln-linear 

ce,n 

2-11 

ln-linear 

ce,na.nd. 

Although  ln-llnear  was  not  always  the 

best  (e.g. , see  selection  2-7)  it  is  very  nearly  as  good  as  the  best  (linear  for 
2-7)  and  so  for  consistency  we  have  used  the  In -linear  model.  Hie  results  are: 


(2-5) 

0£  = 72.19Ct0,4^J0,27 

(7) 

(2-7) 

0E  - 62.93CT0,44//^NA°*2FiND0*034^ 

(«) 

(2-9) 

eE  « 34.91CE0a2^0*049 

(9) 

(2-U) 

- 23.64Ce0*27Na°-076/nd0'091. 

(10) 

These  equations  are  strikingly  similar  in  form  to  tneir  counterparts  developed 
in  ref  (1).  Also,  there  is  remarkable  similarity  in  the  various  "coefficients" 
for  a given  term  (independent  variable)  among  the  sets. 

Example:  Predicting  0^: 

System  No.  2 (randomly  selected)  has 
N = 4C.863 
Na  = 36,581 
Nd  = 10,282 
CT  - 7,741 
C£  ■ 2,196 
©£  ■=  178.15. 
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Equations  (7),  (8),  (9)  and  (10)  lead  to  6e  - 170.11,  170.72,  178.06,  196.51 
respectively. 

Finally,  it  is  interesting  to  use  the  result  (modified  only  to  our  notation)  given 
in  ref.  [1]: 


9. 


5.36  Cp  42Cg64 

l?75* 

nea 


System  No.  2 has  = 57,145  and  Cp  = 5,301  so  that 


6 . (S.  36) (6, 301)1  ~ 4^(2  196>°~ 64 

E (57,J45)1‘*T 


43.58 


(11) 


It  cannot  be  expected  that  the  equation  developed  in  ref  [l]  for  airborne  data 
could  fit  ground  based  data  and  this  result  is  an  illustration  of  this  fact. 

4. 3 Reliability  Gain  Analysis 

Except  possibly  for  certain  basic,  minimal  requirements,  money  is  expended 
on  Reliability  programs  in  order  to  achieve  reliability  gains,  i.e. , increase 
reliability  (which  costs  money)  and  hence  obtain  near  optimal,  or  at  least 
lower,  ll.!e  cycle  coats.  The  possibility  of  doing  this  has  long  been  known  and 
exploited  qualitatively.  In  this  section  we  will  explore  the  relationships 
between  reliability  gain  on  the  one  band,  and  cOBta  on  the  other.  In  particular 
we  will  present  quantitative  relationships  from  which  gain  can  be  assessed. 

The  data  base  for  this  analysis  consists  of  systems  no.  1,  2,  4,  6,  and  10. 
This  fact  requires  some  explanation.  There  are  gains  for  each  phase  (design, 
parts,  evaluation).  This  required  the  calculation  of  a number  of  "initial"  and 
"final"  MTBF's  as  shown  in  Diagram  2.  We  had  to  be  very  particular  about 
the  quality  of  data  in  this  analysis.  Consequently,  enough  knowledge  to  obtain 
the  required  numbers  was  available  only  on  the  i ve  systems  mentioned. 
Actually,  this  small  data  base  is  not  a serious  limitation  because  i)  the  quality 
of  the  data  is  good,  and  ii)  the  number  of  independent  variables  is  small  and 
thus  not  all  of  the  degrees  of  freedom  are  used. 

Before  giving  the  results  of  the  analyses  we  describe  the  calculation  of  the 
various  phase  gains.  All  the  results  of  the  gain  calculations  are  given  in 
Table  4.3.1. 
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TABLE  4.3.1  Reliability  Gain 


1 


\ 

s 


3 

1 

i 


A 


4. 3. 1 Calculation  of  the  phase  gains 

4. 3. 1 . 1 Evaluation  gain 

Generally,  the  final  (achieved)  MTBF,  was  computed  by  fitting  the 
Duane  model  to  the  developmental  data  (for  a complete  description  of  the 
Duane  model  and  fitting  procedures  see  section  3. 1 of  ref.  [12]).  The 
parameter  estimates  were  then  used  to  compute  the  instantaneous  MTBF* 


6 


E 


MTBFj 


(12) 


The  value  of  t used  in  equation  (12)  included  the  reliability  demonstration  tesi 
time.  There  were  two  exceptions  to  this  calculation:  systems  numbered  C 
and  10.  Both  of  these  systems  had  relatively  minor  but  persistent  design 
problems  which  were  not  really  solved  until  after  the  developmental  testing 
(but  before  reliability  demonstration  test).  For  these  two  systems  we  used 
©E  = the  MTBF  observed  during  the  demonstration  test. 

The  MTBF  entering  the  evaluation  phase  (and  hence  leaving  the  purt  phase)  is 
called  Dp  and  was  computed  using  the  fact  that  for  the  Duane  model  the  Initial 
MTBF  is 


0p  » Kr(l/j$  + 1) 


(13) 


where  T (x)  is  the  usual  gamma  function  of  x.  Thus,  the  gain  for  the  evalu 
ation  phase  is 


£/3-U-4 

G = 9 F/0p  - 

E E p $ T(l/(3  + 1) 


(14) 


It  should  be  noted  that  the  Duane  model  was  a good  fit  for  all  five  systems. 
The  values  of  ©E,  ©p  and  G£  are  given  in  Table  4. 3. 1. 

4.3.1. 2  Parts  gain 

A determination  of  ©p,  the  system  reliability  before  the  parts  program 
phase,  was  made  by  defining  the  reliability  improvement  of  the  system  due 
to  the  reliability  parts  program.  The  reliability  parts  programs  for  each  of 
the  five  systems  examined  for  the  gain  analysis  had  similar  parts  program 


*To  use  cumulative  MTBF  would  bias  the  results  low  because  of  the  inclusion 
of  already  corrected  faults. 
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» 

I tasks  performed,  i. e. , parts  standardization  and  vendor  control.  The 

[ identifiable  differences  between  these  similar  parts  program  tasks  were 

| the  levels  of  effort  expended  due  to  the  individual  system  complexity  and 

I part  types.  An  assessment  of  the  similar  parts  program  efforts,  parts 

; standardization  and  /endor  control,  for  each  system  was  made  and  rated  as 

f to  the  effectiveness  of  the  efforts  expended  and  a lactor  was  determined  for 

I the  gain  attributable  to  these  efforts.  This  gain  factor  determined  for  each 

S system  is  as  follows:  for  system  numbers  1,  2,  and  4 the  factor  is  2.5;  for 

| system  number  6 the  factor  is  2.0;  for  system  number  10  the  factor  is  1.5. 

I Parts  screening  was  accomplished  on  two  of  the  five  gain  analysis  systems 

v (system  nos.  1 and  2).  This  effort  consisted  of  subjecting  specific  part 

I types  to  burn-in  which  qualified  these  parts  ”B  level’'  quality. 

« 

Example:  System  number  2 had  1,413  devices  that  were  subjected  to  parts 
screening  out  of  a total  system  parts  count  of  46,  863,  The  improvement  in 
overall  system  reliability  (MTBF),  obtained  by  assessing  the  system  failure 
rate  with  the  screened  parts  compared  to  the  system  failure  rate  had  the 
same  parts  not  been  subjected  to  screening,  was  4.25  hours 


(MTBF  improvement  - - — 

aB  level 


AC  level 


or  an  attributable  parts  phase  gain  of  0.24.  Therefore,  the  total  reliability 
parts  phase  gain  was  2.  74  (2.5  + 0.  24)  yielding  a post  design  MTBF  (©D)  of 
18.01  hours  (49.36/2.74). 

One  other  system,  system  number  6,  had  extensive  qualification  and  analysis 
efforts  expended  on  memory  modules.  This  did  not  increase  the  established 
quality  level  of  the  menory  modules  as  burn-in  did  or  the  parts  of  the  two 
other  systems,  but  rather  was  an  expenditure  cf  effort  apart  from  burn-in 
and  was  approached  differently  in  the  parts  gain  analysis.  System  6 had 
420  memory  devices.  The  percentage  failure  rate  contribution  to  the  total 
system  failure  rate  for  these  particular  devices  yielded  a AMTBF  of  2.42 
hours.  The  assessed  gain  attributable  to  the  effort  expended  was  0.02 
(1.65%  improvement  of  2.42  hours  or  MTBF  improvement  of  0.04  hours). 
This  increased  the  parts  gain  (Gp)  for  system  6 from  2.00  to  2.02  yielding 
a post  design  ,VTrBF  (0p)  of  1.62  hours. 

The  post  design  MTBF  (0p)  and  the  parts  gain  (Gp)  for  each  of  the  five 
systems  are  shown  in  the  Reliability  Gain  Results  table,  Table  4.3.1. 

4.  3. 1. 3 Design  gain 

The  MTBF  leaving  the  design  phase,  ©p,  was  computed  as  described  in  the 
immediately  preceding  subsection  4.3.1. 2.  In  order  to  compute  the  design 
gain  Gp  = ©p/©j  it  is  of  course  necessary  to  calculate  the  initial  (entering) 
MTBF,  ©j.  This  quantity  ©t  is  an  elusive  quantity  to  say  the  very  least:  it 
is  the  MTBF  that  would  be  obtained  if  no  money  (except  that  money  implicitly 
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spent  on  reliability  in  the  course  of  any  program;  this  money  could  never  be 
identified  separately)  were  expended  on  reliability  at  all.  What  we  did  was 
to  carefully  review  each  of  the  design  programs  of  the  five  systems  in  the 
data  base,  particularly  in  the  proposal  phase.  This  effort  was  devoted  to 
identifying  (in  the  proposal  phase)  the  initial  MTBF  prediction,  say  ©predm* 
This  is  the  initial  prediction  without  any  analysis  to  drive  the  design  to  meet 
die  specified  MTBF,  0O  (©Q  = ©spec).  The  results  are: 


System  No. 

epred(i) 

a 

spec 

1 

51.8 

90 

2 

53.4 

125 

4 

194.6 

200 

6 

172.7 

190 

10 

115.0 

200 

We  take  GD  = Q3pec/©pred<i)  “<*  hence 

ei - 8d/gd- 

The  results  are  given  in  Table  4. 3. 1. 

4.  3.2  Results  of  the  gain  analyses 

The  gain  analyses  were  performed  on  variable  sets  3-1  through  3-42.  The 
best  fit  result  are  given  in  Table  4.3.2.  It  is  clear  that  the  ln-linear 
model  is  virtually  always  the  best  (with  respect  to  R)  and  when  it  is  not  it  is 
very  close  to  the  best.  Further  evidence  is  that  in  ref.  (l]  the  ln-linear 
model  was  also  found  to  fit  very  well.  Thus  we  will  use  the  ln-linear  model 
throughout  the  gain  analysis. 

The  important  variable  seta  are: 


Selection  No. 

Dependent  Variable 

Independent  Variable  (s) 

3-5 

gd 

CD 

3-6 

GP 

CP 

3-7 

ge 

CE 

3-8 

gt 

CT 

3-35 

gd 

CD 
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TABLE  4. 3. 2 Best  Fit  Results  for  Reliability  Gain  Analysis 


Selection 

R. 

E. 

R 

3-1 

UN 

1.14 

LNL 

89.86 

3-2 

UN 

0. 10 

LNL 

18.15 

3-3 

LNL 

0.00 

LNL 

1.49 

3-4 

LNL 

0.10 

LNL 

3.66 

3-5 

UN 

1.15 

LNL 

20.97 

3-6 

LNL 

0.79 

LNL 

13.77 

3-7 

LIN 

0.22 

LNL 

69.74 

3-8 

UN 

0.65 

LNL 

90.05 

3-9 

UN 

l.»4 

LNL 

87.31 

3-10 

UN 

1.14 

LNL 

80.63 

3-11 

UN 

0.76 

LNL 

59. 41 

3-12 

UN 

0.80 

LNL 

68.49 

3-13 

LIN 

0.73 

LNL 

44.01 

3-14 

LIN 

0.39 

LNL 

57.59 

3-15 

LIN 

0.48 

LIN 

73.75 

3-16 

UN 

0.33 

LNL 

40.11 

3-17 

LIN 

0.66 

LNL 

85.03 

3-18 

LIN 

0.67 

LNL 

94.28 

3-19 

LNL 

0. 03 

LNL 

6.25 

3-20 

UN 

0.37 

LNL 

73.15 

3-21 

UN 

0.47 

LNL 

97.84 

3-22 

UN 

0.29 

LNL 

47.42 

3-23 

LIN 

0.69 

LNL 

38.56 

3-24 

UN 

0.78 

LNL 

51.36 

3-25 

UN 

0.60 

LNL 

84.10 

3-26 

UN 

0.01 

LNL 

1.26 

3-27 

LIN 

0.00 

UN 

0.87 

3-28 

UN 

0.00 

LIN 

0.89 

3-29 

LNL 

0.09 

LNL 

3.62 

3-30 

LNL 

0.08 

LNL 

3.38 

3-31 

LNL 

0.00 

LNL 

0.82 

3-32 

UN 

1.24 

LNL 

98.40 

3-33 

LNL 

0.05 

LNL 

25.54 

3-34 

LNL 

1.07 

LNL 

11.  73 

3-35 

LNL 

0.70 

LNL 

11.94 

3-36 

LNL 

0.09 

LNL 

4.53 

3-37 

UN 

0.40 

LNL 

31.73 

3-38 

UN 

0.35 

LNL 

72.58 

3-39 

UN 

0. 40 

UN 

73.44 

3-40 

LNL 

0.20 

LNL 

51. 15 

3-41 

LNL 

0.00 

LNL 

3.82 

3-42 

LNL 

0.00 

LNL 

8. 89 
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Selection  No. 

Dependent  Variable 

Independent  Variable  (s) 

3-36 

GP 

GP 

3-37 

ge 

CE 

3-38 

gt 

CT,  N 

3-39 

gt 

CT*  NEA 

3-40 

gt 

CT’  NA*  ND 

3-41 

gt 

CD»  cp.  CE 

3-42 

gt 

CD‘  CP’  CE 

The  independent  variables  Cj^,  Cp,  Cg,  are  the  relative  costs 

CD  = CD//CT 

Cp  = Cp/CT  (16) 


CE  " CE^CT 

where 

C„.  - + C«  + C„ 

T D P E 


and  are  given  in  Table  4. 3. 3 for  all  ten  systems  even  though  only  systems 
1,  2,  4,  6,  and  10  comprise  the  gain  analysis  data  base.  The  relative  costs 
will  be  used  in  the  optimal  allocation  analysis  of  section  4.4. 

4. 3. 2. 1 Total  cost  gain  analysis 

It  is  of  no  particular  importance  that  the  variable  sets  3-38,  3-39  and  3-40 
are  relatively  poor  fits  since  the  major  independent  variables  are  Cj.  Cq, 
Cp  and  C£.  Thus  the  important  sets,  for  Gj,  are  3-8,  3-41  and  3-42. 

The  equation  for  Gj  in  terms  Oi  Cf  is 


G 


T 


0.27 


,0.56 

T 


(17) 
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TABLE  4*3*3  Relative  Reliability  Program  Phase  Costa 


System  Number 

CD 

cp 

CE 

1 

0.0279 

0.6466 

0.3255 

2 

0.0315 

0.8848 

0.2837 

3 

0.0309 

0.6254 

0.3437 

4 

0.0290 

0.6258 

0.34F2 

5 

0.0459 

0.6931 

0.2610 

6 

0.0122 

0.4331 

0. 5547 

7 

0.0735 

0.6973 

0. 2292 

8 

0.0240 

0.6837 

0.2923 

9 

0.1109 

0. 5106 

0.3785 

10 

0.0399 

0.3909 

0. 5692 

MEAN 

0.0426 

0. 5991 

0.3583 

This  fit  is  not  very  good  but  the  fit  for  G-j-  in  terms  of  the  individual  absolute 
and  relative  costs  are  much  better;  indeed  quite  good: 


Gt  = (2.31  x 10“U)C^  44  C"3, 61  Cg  01 


(18) 


c + c + c = c 


Gt  - <2.25  x 10-6)CM-1.37)cM-10.39)c^(-5.88) 


(19) 


= 1 


This  fact  illustrates  that  while  total  costs  affect  total  gain,  muah  more 
important  is  how  the  money  (measured  in  man-days  in  this  report)  is  spent. 
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We  defer  an  example  until  we  have  discussed  the  Individual  gain  results 
by  phase. 

4. 3. 2. 2 Design  cost  gain  analysis 

For  this  analysis  the  important  variable  sets  are  3-5  and  3-35.  These  two 
sets  are  good  fits  and  the  results  are: 


Gd  = 0.27  34 


(20) 


Gd  = 9.42  cjj(0*48) 


o < cjj  ^ (1  - Cp  - c|_) . 


(21) 


The  result  of  equation  (20)  is  strikingly  similar  to  the  result  (in  our  notation) 
found  in  ref.  [1]: 

°D  ■ °-30  CD  25  • 

No  results  were  given  in  ref.  [l]  to  permit  comparison  with  equation  (21). 

4. 3. 2. 3 Part  cost  gain  analysis 

The  important  variable  sets  here  are  3-6  and  3-36,  Again,  the  fits  were 
quite  good  and  the  results  are: 


Gp  = 0. 19  Cp  23  (22) 

Gp  - 3.87  C^°'91'  , 0 < Cp  ^ (1  - Cjj  - Cg),  (23) 

The  result  of  equation  (22)  is  somewhat  similar  to  the  result  in  ref.  [l], 
which  in  our  notation  is: 


1.145 


r0. 137 
UP 


As  before,  and  also  for  GE,  no  results  were  given  in  ref.  [1)  to  permit 
comparison  to  equation  (23).  That  is,  relative  costs  were  not  investigated 
in  ref.  f 1 ]. 
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4. 3. 2. 4 Evaluation  cost  gain  analysis 

The  important  data  sets  are  3-7  and  3-37  and  the  results  are: 

1 fil 

G£  = 0.000029  Cg  (24) 

Ge  = 561.94  ce(4*20)  0 < CE  £ (1  - JD  - . (25) 

Although  the  fit  was  not  very  good  for  equation  (24)  (the  R was  quite  large) 
it  is  similar  to  die  result  given  in  ref.  U ] (in  our  notation) 

Ge  = 0.0064  C®* 96  . 

The  results  of  these  analyses  are  shown  graphically  in  Figure  4.3. 1. 

Example 
It  is  clear  that 


0D\  /®p\  (?e\ 

Pi)  W Vep; 


GDGpGE  . 


(26) 


Thus,  it  will  be  interesting  to  compare  the  (direct)  results  obtained  from 
equation  (18): 


CT  - 2.31  x 10'UC®,44Cj?,61c|-01 


and  equations  (20),  (22)  and  (24): 

Gd  = 0.27  cj,-34;  Gp  = 0.19  C9,29;  G£  = 0.000029  C*’61 


System  No.  4 (randomly  selected)  has 

CD  = 207,  Cp  = 4,467;  C£  = 2,464. 
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Thus,  from  equation  (18)  we  have 


Gt  - 20.45, 


From  the  individual  equations 

Qd  - 1.86;  Gp  * 2.1?;Ge  « 8.37 


and 

Gt  = 30.15. 


Both  of  these  calculated  values  of  Gq-  compare  well  with  the  actual  (observed) 
gain  for  system  no.  4 of  22.  82.  From  a purely  mathematical  standpoint, 
equation  (18)  would  be  the  preferred  method, 

4.4  Optimal  Allocation  Analysis 

A question  of  some  importance  in  allocating  reliability  expenditures  is:  for  a 
fixed  total  amount  of  expenditure  how  should  the  effort  be  divided  among  the 
phases?  That  is,  for  fixed  Cq-,  what  should  Cp,  Cp,  and  Cr  be  where 
CD  + Cp  + CE  = CT.  Since  Cq-  itself,  varies  from  program lo  program,  and 
since 


it  is  sufficient  to  optimally  allocate  to  the  relative  costs  cj),  Cp  and  Cg. 
Then  for  a fixed  and  known  Cq-  the  values  CD,  Cp  and  CE  can  be  obtained 
from  Cpj  = CjjCq-;  Cp  — CpCq<;  CE  — CECq>. 


To  keep  things  simgle  we  will  use  the  linear  model.  It  has  a very  small  R.  E. 
and  a not  too  large  R so  it  is  a reasonable  fit.  We  will  address  the  best 
fitting  case,  In-linear,  shortly. 

For  the  linear  model 


CT  = 90.36  - 3,569.25  C'D  - 94.00  Cp  + 309.88  Cg.  (27) 
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However,  since  Cp  + Cp  + Cg  " 1,  one  of  the  variable*  can  be  eliminated. 
Arbitrarily  eliminating  cjg  we  have 

0T  - 400. 24  - 3,  879. 13  c{j  - 403.  88  Cp.  (28) 


Of  courae  it  ie  silly,  even  incompetent,  to  allow  absurdities  like  Cp  « 0, 

Cp  * o,  Cp  - 1 or  CL  = 1,  Cp  - 0,  Cg  = 0.  There  are  many  other  absurdi- 
ties like  these  and  in  order  to  avoid  them  we  need  additional  restrictions. 
Using  the  observed  values  actually  expended  from  Table  4. 3. 3 we  establish 
the  following  additional  restrictions.  Generally  we  took  (near)  the  maximum 
and  minimum  observed  expenditure  rates. 

0.01  4 Cp  4 0. 12\ 


0.35  4 Cp  4 0.70  \ (29) 


0.20  4 Cg  4 0.61' 


CD  + CP  + CE  “ 1 * 


Now  we  can  view  the  problem:  find  Cp,  Cp  and  Cg  so  that  GT  of  equation  (28) 
is  maximized  subject  to  the  restrictions  of  (29),  as  a linear  programming 
problem.  Actually,  this  problem  could  be  solved  graphically  but  there  are 
many  "canned"  linear  program  routines.  We  used  the  LINPRO  routine 
written  in  XBAS1C  for  the  G.  E.  265  computer.  The  results  are: 


Cq  = 0.01 

Cp  = 0.39 

Cg  * 0.60 


and 


Gt  = 203.94. 


The  region  of  feasible  solutions  and  the  optimal  solution  are  shown  in 
Figure  4.4. 1. 
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Th*  solution  is:  spend  ss  littls  ss  possible  on  reliability  dsalan  costs  and  ss 
much  as  possible  on  rs liability  evaluation.  This  is  to  bs  expsetsd  ainos  tbs 
grsatsat  galas  wars  aohisvsd  in  tin  evaluation  phase.  Ons  might  bs  tempted 
to  think  tbs  solution  is  not  very  good  because  of  the  old  phrase  "it  ia  batter 
to  build  quality  ia  rather  than  i aspect  it  ia";  however,  it  must  be  remembered 
Cp  represents  reliability  design  expenditure  and  not  total  design  expenditure. 
Clearly,  from  a reliability  viewpoint,  the  expenditures  are  relatively  more 
efficiently  made  in  the  evaluation  phase. 

Now  we  address  the  In-line ar  modal  for  Gy  since  it  is  the  beat  fitting  modal. 
The  linear  program  approach  will  not  work  because,  even  though  we  can 
transform  to  linearity  in  the  logarithm  the  restrictions  given  in  (29)  become 
non-linear.  However,  one  of  the  variables  can  be  eliminated  from  the 
equation  for  G*j-  since,  as  before,  Cp  + Cp  + Cg  ■ 1 and  we  have 


Qt  - (2.25  x 10"®) <1  - Cp  - Cg)"1*  3TCp("l°-  39)CM“6*  88>. 


In  Figure,  4.4.2  we  have  shown  '**•]-  is  a function  of  Cg  for  various  values  of 
Cp.  The  solid  lines  represent  yUuef  of  that  are  admissible  in  the  sense 
that  the  independent  variables  Cd.  Cp,  Ce  satisfy  dm  restrictions  of  (29).  It 
is  clear  from  Figure  4. 4.2,  that  while  we  have  not  found  die  absolute  opti- 
mal solution,  the  optimal  solution  is  near  Cq  * 0.01;  Cp  = 0.42;  cjg  » 0.57. 
This  is  about  the  same  as  the  solution  for  the  linear  case  previously 
discussed.  There  the  optimal  solution  was  c[>  ■ 0.01;  Cp  =>  0.39;  Cg  = 0.60. 

4. 5 Prediction  of  Individual  Phase  MTBF's 


In  this  section  we  will  give  the  equations  for  predicting  the  individual  phase 
MTBF's  based  on  just  two  independent  variables  in  each  case:  the  reliability 
expenditures  in  that  phase  and  the  MTBF  entering  that  phase.  The  in-linear 
models  are  the  overall  best  fitting  models  for  the  three  variables  sets  3-2, 
3-3,  and  3-4.  It  should  be  noted  that  Ojy,  Op  and  Og  are  the  absolute  values 
of  the  MTBF  leaving  a particular  phase  given  that  the  previous  phase,  if  any, 
has  been  implemented.  If,  for  example,  the  parts  phase  had  not  been  imple- 
mented (hen  the  ratio  (from  this  study)  Og/Op  = 9*  (say)  could  be  applied  to 
Od  (assuming  the  design  phase  was  implemented)  to  determine  Bg,  i.e. 

Og  - 9*0 D.  In  short,  omission  of  a phase  will  affect  the  absolute  value  of 
the  MTBF  leaving  subsequent  phases. 

4. 5. 1 Prediction  of  ©D 

In  this  case  the  independent  variables  are  Cp  and  Op  the  ln-iinear  model  fits 
well  (see  Table  4. 3. 2)  and  we  have 


SS 


4.5.2  Prediction  of  0p 

The  ln-Hnear  model  is  an  excellent  fit  {see  Table  4.3.2)  and  we  obtain 


e 


P 


0.199 


1. 15„0. 26 
D '“P  ' 


4.5.3  Prediction  of 

The  ln-linear  model  is  again  an  excellent  fit  (see  Table  4. 3. 2)  and  we  obtain 


E 


18.64  9 


0.014^0.29 
P CE  * 


Example;  System  No.  10  (randomly  selected)  has 
CE  = 2, 110;  9p  ■ 5. 16;  ©E  = 173 

thus 

9e  = 18. 64  (8. 16)°* 014  (2,U0)0,29 
= 174.9. 


Using  the  model  from  ref.  [1)  for  final  MTBF  (changed  only  to  fit  our 
notation) 


©E  = 0.094(5.16)°-68(2,110)0,74 
= 83.8. 


Of  course,  we  would  not  expect  the  airborne  model  to  fit  the  ground  base 
model. 


57 


SECTION  5.0  CONCLUSIONS  AND  RECOMMENDATIONS 

Based  on  the  results  obtained  in  ref.  [ l]  for  airborne  systems  and  this 
present  investigation  for  ground  baaed  systems,  the  relationships  between 
reliability  cost  and  reliability  achieved  can,  and  have  been  quantified.  Also, 
the  models  developed  in  this  present  study,  in  most  cases,  are  quite  excellent 
fits  (this  is  not  to  say  the  models  developed  in  ref.  [l]  were  not  good  fits) . Of 
course  we  have  for  the  reader  the  usual  admonition  that  use  of  the  models 
outside  the  range  of  the  independent  variables  in  the  data  base  is  risky.  How- 
ever, the  risk  should  be  minimal  since  we  know  that  the  variables  in  question 
are  related  in  some  fashion. 

[ Although  the  comparisons  we  made  show  that  the  airborne  models  from  ref.  M 

cannot  be  used  for  tne  ground  based  data,  the  resemblance  between  the  various 
i models  of  the  two  studies  was  striking  in  the  gain  analysis.  This  lends  further 

i credence  to  the  validity  of  the  relationships  and  models  developed. 

Finally,  all  of  the  models  are  worth  developing  further,  i.e. , by  expanding 
f the  data  base.  However,  for  special  mention  we  single  out  the  optimal  alloca- 

tion analysis  of  section  4.4.  The  data  base  should  be  expanded  and  the  optimal 
allocation  should  be  studied  in  much  more  detail  since  it  is  a powerful,  very 
powerful,  tool. 
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Pradictad  Total  Coat 


PLgura  6.1  Pradlotad  Total  Coat  ts  Actual  Total  Cost 


Predicted  Total  Cost 


Figure  6.?  Predicted  Total  Cost  re  Actual  Total  Coot 
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Figure  6.6  Predicted  Bvmluation  Ruse  Cost  re  Actual  Braluatioc  Ruse  Coat 
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Figure  6.19  Predicted  Evaluation  Gain  vs  Actual  Evaluation  Gain 
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MISSION 

of 

Rome  Air  Development  Center 


RADC  is  the  principal  AFSC  organization  charged  with 
planning  and  executing  the  USAS'  cxploratoi  y and  advanced 
development  programs  for  information  sciences , intell  ~ 
gence , command,  control  and  communications  technology, 
products  and  services  oriented  to  the  needs  of  the  USAF . 
Primary  RADC  mission  areas  or s communications,  electro- 
magnetic guidance  and  control,  surveillance  of  ground 
and  aerospace  objects,  intelligence  data  collection  and 
handling,  information  system  technology,  and  electronic 
reliability,  maintainability  and  compatibility.  RADC 
has  mission  responsibility  as  assigned  by  AFSC  for  de- 
monstration and  acquisition  of  selected  subsystems*  and 
systems  in  the  intelligence,  mapping,  charting,  command, 
control  and  communications  are*.s. 


